Conducting Probabilistic Sensitivity Analysis for Decision Models Based on Markov Chains

Models based on Markov chains are used for decision-making studies in many application domains; however , there are no widely adopted methods for performing sensitivity analysis on the associated transition probability matrices (TPMs). This article describes two simulation-based approaches for probabilistic sensitivity analysis of a given finite-state, finite-horizon, discrete-time Markov chain with a stationary TPM and an uncertainty set within which the chain's TPM may vary according to an appropriate probability distribution. Both approaches sample each row of the TPM independently. The first approach assumes no prior knowledge of the TPM's distribution, and each row is sampled uniformly over its uncertainty set. The second approach involves random sampling from the (truncated) multivariate normal distribution of the TPM's maximum likelihood estimator subject to the condition that each row has nonnega-tive elements, and sums to one. The proposed methods are easy to implement and have reasonable computation times. As an illustrative example, these two methods are applied to a medical decision-making problem involving the evaluation of treatment guidelines for glycemic control of patients with type 2 diabetes in which the natural variation in glycated hemoglobin (HbA1c) is modeled as a Markov chain.